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DETAILED ACTION 

This communication is responsive to Amendment, filed 05/20/08. 

Claims 1-2, 7-23, 28-33, 37-38, 40-41, 44-46 are pending in this application. 
Claims 1, 22, 37, 38, 40-41 are independent claims. This action is made Final. 

The rejection of claims 37, 38 by 35 U.S.C. §101 has been withdrawn in view of 
the amendment. 

Claim Rejections - 35 USC §112 

The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

Claims 43-46 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

The term "adjacent" in claims 43-46 is a relative term which renders the claim 
indefinite. The term "adjacent" is not defined by the claim, the specification does 
not provide a standard for ascertaining the requisite degree, and one of ordinary 
skill in the art would not be reasonably apprised of the scope of the invention. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 1 03(a), the examiner presumes that the subject matter of the 
various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the 
obligation under 37 CFR 1 .56 to point out the inventor and invention dates of each 
claim that was not commonly owned at the time a later invention was made in order 
for the examiner to consider the applicability of 35 U.S.C. 1 03(c) and potential 35 
U.S.C. 1 02(e), (f) or (g) prior art under 35 U.S.C. 1 03(a). 

Claims 1, 2, 7-10, 12-15, 17, 18, 22, 23, 28-31, 33, 37, 38, 40, 41, 43-46 are 
rejected under 35 U.S.C. 103(a) as being unpatentable over Pan et al. (US Patent 
No. 6,993,246), in view of Lyle (US Patent No. 6,470,359), and further in view of 
Nunally et al. (US Patent No. 6,035,341). 

As per claim 1, Pan teaches a method of storing data, said method comprising 
the steps of: 

storing data (i.e. video and audio data is stored on a hard disk, col. 4, lines 58- 
65), as one or more data samples (i.e. a sequence of video and audio frames, col. 3, 
lines 20-27; event e1, ... event e6; event f1 .... event f5; See Fig. 2), in one or more 
media files (i.e. Data stream #1, Data stream #2 in Fig. 2; a digital file, col. 1, line 59 to 
col. 2, line 5) configured for use by a media player application (i.e. a media file that can 
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be played back through an application such as Microsoft Windows Media Player, col. 3, 
lines 20-27) in playing the data samples (i.e. a "data stream" is a sequence of video and 
audio frames read from a media file that can be played back through an application 
such as Microsoft Windows Media Player, col. 3, lines 20-27); 

storing, in an index file (i.e. a data set is maintained in an index file, col. 2, lines 
11-26) associated with one or more the media files, at least an offset value (i.e. 
positional information, col. 1, line 59 to col. 2, line 5) for each of the data samples 
representing a location (i.e. positional information indicating where data associated with 
the events are located in a corresponding storage medium such as a digital file or where 
the data is located within the data stream, col. 1, line 59 to col. 2, line 5) of each of the 
data samples in a corresponding one of the media files, each of the media files further 
comprising additional information (i.e. data pointer, time, See Fig. 2) interspersed 
throughout that media file (i.e. correlating data among multiple data streams based on a 
use of time-stamps and related positional information, col. 1, lines 46-58; See Fig. 2), 
the additional information comprising at least a timestamp (i.e. the time-stamps are 
used as an index to locate where the data associated with the corresponding event is 
stored in the second data stream, col. 2, lines 27-40) for each of the data samples (i.e. 
event e1, ... event e6; event f1 .... event f5; See Fig. 2), each of the timestamps (i.e. t1, 
t3, t10, t6, t8, t11, t2, t4, t5, t7, t9; See Fig. 2) indicating a capture time of an associated 
data sample (i.e. During a presentation of information at slide display device 580, an 
audio-video recording unit 585 captures details of, for example, a corresponding slide 
presentation including a lecturer delivering a speech. A data stream generated by the 
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audio-video recording unit 585 is captured for storage in a data file at data processing 
unit 560, col. 10, lines 39-51). 

It should be noted that Pan teaches the additional information of the media file is 
used in constructing the index file as shown in Fig. 2 (i.e. a data set is maintained in 
an index file, col. 2, lines 11-26; The resulting event-data pointer record 152, 
(E.sub.i,j,P.sub.i,j,T.sub.i,j), is then stored in a data stream index file #1 235. There is 
usually one index file for each data stream, Pan, col. 4, lines 34-39), wherein the data 
set of Pan comprises Event E (data sample of the claimed limitation), Pointer P 
(additional information of the claimed limitation), Timestamp T (additional information of 
the claimed limitation). 

The index file of Pan further comprises the offset values (i.e. positional 
information, col. 1, line 59 to col. 2, line 5) representing the location of each of the data 
samples in the media files (i.e. positional information indicating where data associated 
with the events are located in a corresponding storage medium such as a digital file or 
where the data is located within the data stream, col. 1, line 59 to col. 2, line 5). 

Pan does not explicitly teach: 

the additional information of the media files being used in reconstructing the 
index file upon corruption of the index file, the reconstructed index file comprising the 
offset values representing the location of each of the data samples in the media files, 
wherein the reconstructed index file replaces the index file associated with the one or 
more media files. 

Lyle teaches: 
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the additional information (i.e. additional information within a space map page in 
a LOB table space, col. 4, line 60 to col. 5, line 6) of the media files (i.e. LOB, Large 
objects, such as image data, col. 2, lines 27-37) being used in reconstructing the index 
file (i.e. to recover an index on an auxiliary table by reading only the LOB low-level 
space map pages, col. 4, line 60 to col. 5, line 6) upon corruption of the index file (i.e. 
the index could be corrupted, col. 2, lines 38-44), the reconstructed index file comprising 
the offset values (i.e. the index recovery system 124 indicates for each page in the 
space map page whether that page is the first page allocated to a LOB, col. 4, line 60 to 
col. 5, line 6) representing the location of each of the data samples in the media files 
(i.e. The index recovery system 124 of the present invention includes additional 
information within a space map page in a LOB table space. In addition to recording 
whether a page is allocated or deallocated, the index recovery system 124 indicates for 
each page in the space map page whether that page is the first page allocated to a 
LOB. Storing this information with a LOB low-level space map page enables the index 
recovery system 124 to recover an index on an auxiliary table by reading only the LOB 
low-level space map pages, instead of all pages in the LOB table space, col. 4, line 60 
to col. 5, line 6). 

It should be noted that the claimed invention discloses using additional 
information such as image information to reconstruct the index file (i.e. said image 
information allows for reconstruction of said index file upon corruption thereof, See 
Application, [0017]). Similarly, Lyle teaches using of additional information such as 
image information to rebuild the index. 
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It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan and Lyle at the time the invention was made to modify the system of Pan to 
include the limitations as taught by Lyle. One of ordinary skill in the art would be 
motivated to make this combination in order to recover an index on an auxiliary table in 
view of Lyle (col. 4, line 60 to col. 5, line 6), as doing so would give the added benefit of 
enabling the index recovery system 124 to recover an index on an auxiliary table by 
reading only the LOB low-level space map pages, instead of all pages in the LOB table 
space as taught by Lyle (col. 4, line 60 to col. 5, line 6). 

Pan, Lyle do not explicitly teach the reconstructed index file replaces the index 
file associated with the one or more media files. 

Nunally teaches the step of updating the index information for video and audio 
files (i.e. the sequence is added to a positive result list (step 152) and the index 
information for the file is updated to indicate detection of the event (step 154). That is, 
the event-related data shown at 104 in FIG. 5 is updated to indicate detection of the 
event, as well as the confidence factor applicable to the event detection decision, col. 
10, lines 37-62; FIG. 5 illustrates a format in which compressed video and audio data 
are stored on one or more of the hard disk drives of the analysis/storage device. As 
seen from FIG. 5, the data stored on the hard drives includes compressed video and 
audio data files indicated generally at 92 and index data indicated generally at 94, col. 
8, line 66 to col. 9, line 14). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally at the time the invention was made to modify the system of Pan, 
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Lyle to include the limitations as taught by Nunally. One of ordinary skill in the art would 
be motivated to make this combination in order to update the index information for video 
and audio files in view of Nunally (col. 10, lines 37-62), as doing so would give the 
added benefit of providing an integrated video and audio recording device with 
advanced information management capabilities as taught by Nunally (col. 1, lines 37- 
39). 

As per claim 22, Pan teaches a method of storing video (i.e. video frames, col. 
11, lines 45-52) and associated text data (i.e. textual data stream, col. 11, lines 16-34), 
said method comprising the steps of: 

storing the video (i.e. video and audio data is stored on a hard disk, col. 4, lines 
58-65) associated text data (i.e. textual data stream, col. 11, lines 16-34), as one or 
more data samples (i.e. a sequence of video and audio frames, col. 3, lines 20-27; 
event e1, ... event e6; event f1 .... event f5; See Fig. 2), in one or more media files in 
accordance with first file format (i.e. closed captions for video or audio content, col. 11, 
lines 15-23; Another aspect of the present invention involves correlating more than two 
data streams and storing related time-stamped event data pointers to a storage device 
for later retrieval. As previously mentioned, the time-stamps indicate a reference time 
when a particular event was detected while a corresponding data pointer indicates a 
location where data associated with that event was stored in the data file, col. 2, lines 
58-64), each media file being configured for use by a media file player application in 
playing the video data (i.e. video playback system, col. 11, lines 45-52); 
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storing, in a index file (i.e. a data set is maintained in an index file, col. 2, lines 
11-26) in accordance with a second file format (i.e. audio samples, col. 11, lines 45-52), 
at least an offset value for each of the data samples (i.e. a sequence of video and audio 
frames, col. 3, lines 20-27; event e1, ... event e6; event f1 .... event f5; See Fig. 2) 
representing a location of each of the one or more data samples in a corresponding one 
of the media files (i.e. Another aspect of the present invention involves correlating more 
than two data streams and storing related time-stamped event data pointers to a 
storage device for later retrieval. As previously mentioned, the time-stamps indicate a 
reference time when a particular event was detected while a corresponding data pointer 
indicates a location where data associated with that event was stored in the data file, 
col. 2, lines 58-64); and 

adding additional information (i.e. data pointer, time, See Fig. 2) interspersed 
throughout each of the media files (i.e. correlating data among multiple data streams 
based on a use of time-stamps and related positional information, col. 1, lines 46-58), 
the media files including the additional information being readable by a media player 
application (i.e. a media file that can be played back through an application such as 
Microsoft Windows Media Player, col. 3, lines 20-27) corresponding at least to first file 
format (i.e. video playback system ... a standard media player, col. 11, lines 45-52), the 
additional information comprising at least a timestamp (i.e. the time-stamps are used as 
an index to locate where the data associated with the corresponding event is stored in 
the second data stream, col. 2, lines 27-40) for one or more of the data samples (i.e. a 
sequence of video and audio frames, col. 3, lines 20-27; event e1, ... event e6; event f1 
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.... event f5; See Fig. 2), each of the timestamps indicating a capture time (i.e. t1, t3, 
t10, t6, t8, t11, t2, t4, t5, t7, t9; See Fig. 2) of an associated data sample (i.e. The 
principles of the present invention have applications beyond note taking, text entry, or a 
close captioning system. One example is a video surveillance system for an automobile 
parking lot. This example system uses multiple video cameras. One set of cameras 
record the automobiles and associated license plates at the entrance and exits. Another 
set of cameras record the parking area. In current video surveillance systems, cameras 
continuously record video on tape. This results in a waste of video recording media. A 
more efficient system would record, for example, only for a few seconds after a motion 
detector indicates when a car has entered or left the garage or parking space. This type 
of sporadic recording renders it difficult to determine where an event is recorded on a 
videotape. Even a time-stamped videotape must be searched for the appropriate time 
match, col. 13, line 62 to col. 14, line 9). 

It should be noted that Pan teaches the additional information of the media file is 
used in constructing the index file as shown in Fig. 2 (i.e. a data set is maintained in an 
index file, col. 2, lines 11-26; The resulting event-data pointer record 152, 
(E.sub.i,j,P.sub.i,j,T.sub.i,j), is then stored in a data stream index file #1 235. There is 
usually one index file for each data stream, Pan, col. 4, lines 34-39), wherein the data 
set of Pan comprises Event E (data sample of the claimed limitation), Pointer P 
(additional information of the claimed limitation), Timestamp T (additional information of 
the claimed limitation). 
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The index file of Pan further comprises the offset values (i.e. positional 
information, col. 1, line 59 to col. 2, line 5) representing the location of each of the data 
samples in the media files (i.e. positional information indicating where data associated 
with the events are located in a corresponding storage medium such as a digital file or 
where the data is located within the data stream, col. 1, line 59 to col. 2, line 5). 

Pan does not explicitly teach: 

the additional information of the media files being used in reconstructing the 
index file upon corruption of the index file, the reconstructed index file comprising the 
offset values representing the location of each of the data samples in the media files, 
wherein the reconstructed index file replaces the index file associated with the one or 
more media files. 

Lyle teaches: 

the additional information (i.e. additional information within a space map page in 
a LOB table space, col. 4, line 60 to col. 5, line 6) of the media files (i.e. LOB, Large 
objects, such as image data, col. 2, lines 27-37) being used in reconstructing the index 
file (i.e. to recover an index on an auxiliary table by reading only the LOB low-level 
space map pages, col. 4, line 60 to col. 5, line 6) upon corruption of the index file (i.e. 
the index could be corrupted, col. 2, lines 38-44), the reconstructed index file comprising 
the offset values (i.e. the index recovery system 124 indicates for each page in the 
space map page whether that page is the first page allocated to a LOB, col. 4, line 60 to 
col. 5, line 6) representing the location of each of the data samples in the media files 
(i.e. The index recovery system 124 of the present invention includes additional 
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information within a space map page in a LOB table space. In addition to recording 
whether a page is allocated or deallocated, the index recovery system 124 indicates for 
each page in the space map page whether that page is the first page allocated to a 
LOB. Storing this information with a LOB low-level space map page enables the index 
recovery system 124 to recover an index on an auxiliary table by reading only the LOB 
low-level space map pages, instead of all pages in the LOB table space, col. 4, line 60 
to col. 5, line 6). 

It should be noted that the claimed invention discloses using additional 
information such as image information to reconstruct the index file (i.e. said image 
information allows for reconstruction of said index file upon corruption thereof, See 
Application, [0017]). 

Similarly, Lyle teaches using of additional information such as image information 
to rebuild the index. 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan and Lyle at the time the invention was made to modify the system of Pan to 
include the limitations as taught by Lyle. One of ordinary skill in the art would be 
motivated to make this combination in order to recover an index on an auxiliary table in 
view of Lyle (col. 4, line 60 to col. 5, line 6), as doing so would give the added benefit of 
enabling the index recovery system 124 to recover an index on an auxiliary table by 
reading only the LOB low-level space map pages, instead of all pages in the LOB table 
space as taught by Lyle (col. 4, line 60 to col. 5, line 6). 
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Pan, Lyle do not fairly teach the reconstructed index file replaces the index file 
associated with the one or more media files. 

Nunally teaches the step of updating the index information for video and audio 
files (i.e. the sequence is added to a positive result list (step 152) and the index 
information for the file is updated to indicate detection of the event (step 154). That is, 
the event-related data shown at 104 in FIG. 5 is updated to indicate detection of the 
event, as well as the confidence factor applicable to the event detection decision, col. 
10, lines 37-62; FIG. 5 illustrates a format in which compressed video and audio data 
are stored on one or more of the hard disk drives of the analysis/storage device. As 
seen from FIG. 5, the data stored on the hard drives includes compressed video and 
audio data files indicated generally at 92 and index data indicated generally at 94, col. 
8, line 66 to col. 9, line 14). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally at the time the invention was made to modify the system of Pan, 
Lyle to include the limitations as taught by Nunally. One of ordinary skill in the art would 
be motivated to make this combination in order to update the index information for video 
and audio files in view of Nunally (col. 1 0, lines 37-62), as doing so would give the 
added benefit of providing an integrated video and audio recording device with 
advanced information management capabilities as taught by Nunally (col. 1, lines 37- 
39). 
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As per claim 37, Pan teaches an apparatus for storing data, said apparatus 
comprising of: 

media file generation means for storing data (i.e. video and audio data is stored 
on a hard disk, col. 4, lines 58-65), as one or more data samples (i.e. a sequence of 
video and audio frames, col. 3, lines 20-27; event e1, ... event e6; event f1 .... event f5; 
See Fig. 2), in one or more media files (i.e. Data stream #1, Data stream #2 in Fig. 2; a 
digital file, col. 1, line 59 to col. 2, line 5) configured for use by a media player (i.e. a 
media file that can be played back through an application such as Microsoft Windows 
Media Player, col. 3, lines 20-27) application in playing the data samples (i.e. a "data 
stream" is a sequence of video and audio frames read from a media file that can be 
played back through an application such as Microsoft Windows Media Player, col. 3, 
lines 20-27); and 

index file generation means for storing, in an index file (i.e. a data set is 
maintained in an index file, col. 2, lines 11-26) associated with one or more the media 
files, at least an offset value (i.e. positional information, col. 1, line 59 to col. 2, line 5) for 
each of the data samples representing a location (i.e. positional information indicating 
where data associated with the events are located in a corresponding storage medium 
such as a digital file or where the data is located within the data stream, col. 1, line 59 to 
col. 2, line 5) of each of the data samples in a corresponding one of the media files, 
each of the media files further comprising additional information (i.e. data pointer, time, 
See Fig. 2) interspersed throughout that media file (i.e. correlating data among multiple 
data streams based on a use of time-stamps and related positional information, col. 1, 
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lines 46-58; See Fig. 2), the additional information comprising at least a timestamp (i.e. 
the time-stamps are used as an index to locate where the data associated with the 
corresponding event is stored in the second data stream, col. 2, lines 27-40) for each of 
the data samples (i.e. event e1, ... event e6; event f1 .... event f5; See Fig. 2), each of 
the timestamps (i.e. t1, t3, HO, t6, t8, t11, t2, t4, t5, t7, t9; See Fig. 2) indicating a 
capture time of an associated data sample (i.e. During a presentation of information at 
slide display device 580, an audio-video recording unit 585 captures details of, for 
example, a corresponding slide presentation including a lecturer delivering a speech. A 
data stream generated by the audio-video recording unit 585 is captured for storage in a 
data file at data processing unit 560, col. 10, lines 39-51). 

It should be noted that Pan teaches the additional information of the media file is 
used in constructing the index file as shown in Fig. 2 (i.e. a data set is maintained in an 
index file, col. 2, lines 11-26; The resulting event-data pointer record 152, 
(E.sub.i,j,P.sub.i,j,T.sub.i,j), is then stored in a data stream index file #1 235. There is 
usually one index file for each data stream, Pan, col. 4, lines 34-39), wherein the data 
set of Pan comprises Event E (data sample of the claimed limitation), Pointer P 
(additional information of the claimed limitation), Timestamp T (additional information of 
the claimed limitation). 

The index file of Pan further comprises the offset values (i.e. positional 
information, col. 1, line 59 to col. 2, line 5) representing the location of each of the data 
samples in the media files (i.e. positional information indicating where data associated 
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with the events are located in a corresponding storage medium such as a digital file or 
where the data is located within the data stream, col. 1, line 59 to col. 2, line 5). 
Pan does not explicitly teach: 

the additional information of the media files being used in reconstructing the 
index file upon corruption of the index file, the reconstructed index file comprising the 
offset values representing the location of each of the data samples in the media files, 
wherein the reconstructed index file replaces the index file associated with the one or 
more media files. 

Lyle teaches: 

the additional information (i.e. additional information within a space map page in 
a LOB table space, col. 4, line 60 to col. 5, line 6) of the media files (i.e. LOB, Large 
objects, such as image data, col. 2, lines 27-37) being used in reconstructing the index 
file (i.e. to recover an index on an auxiliary table by reading only the LOB low-level 
space map pages, col. 4, line 60 to col. 5, line 6) upon corruption of the index file (i.e. 
the index could be corrupted, col. 2, lines 38-44), the reconstructed index file comprising 
the offset values (i.e. the index recovery system 124 indicates for each page in the 
space map page whether that page is the first page allocated to a LOB, col. 4, line 60 to 
col. 5, line 6) representing the location of each of the data samples in the media files 
(i.e. The index recovery system 124 of the present invention includes additional 
information within a space map page in a LOB table space. In addition to recording 
whether a page is allocated or deallocated, the index recovery system 124 indicates for 
each page in the space map page whether that page is the first page allocated to a 
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LOB. Storing this information with a LOB low-level space map page enables the index 
recovery system 124 to recover an index on an auxiliary table by reading only the LOB 
low-level space map pages, instead of all pages in the LOB table space, col. 4, line 60 
to col. 5, line 6). 

It should be noted that the claimed invention discloses using additional 
information such as image information to reconstruct the index file (said image 
information allows for reconstruction of said index file upon corruption thereof, See 
Application, [0017]). 

Similarly, Lyle teaches using of additional information such as image information 
to rebuild the index. 

Thus, it would have been obvious to one of ordinary skill of the art having the 
teaching of Pan and Lyle at the time the invention was made to modify the system of 
Pan to include the limitations as taught by Lyle. One of ordinary skill in the art would be 
motivated to make this combination in order to recover an index on an auxiliary table in 
view of Lyle (col. 4, line 60 to col. 5, line 6), as doing so would give the added benefit of 
enabling the index recovery system 124 to recover an index on an auxiliary table by 
reading only the LOB low-level space map pages, instead of all pages in the LOB table 
space as taught by Lyle (col. 4, line 60 to col. 5, line 6). 

Pan, Lyle do not fairly teach the reconstructed index file replaces the index file 
associated with the one or more media files. 

Nunally teaches the step of updating the index information for video and audio 
files (i.e. the sequence is added to a positive result list (step 152) and the index 
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information for the file is updated to indicate detection of the event (step 154). That is, 
the event-related data shown at 104 in FIG. 5 is updated to indicate detection of the 
event, as well as the confidence factor applicable to the event detection decision, col. 
10, lines 37-62; FIG. 5 illustrates a format in which compressed video and audio data 
are stored on one or more of the hard disk drives of the analysis/storage device. As 
seen from FIG. 5, the data stored on the hard drives includes compressed video and 
audio data files indicated generally at 92 and index data indicated generally at 94, col. 
8, line 66 to col. 9, line 14). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally at the time the invention was made to modify the system of Pan, 
Lyle to include the limitations as taught by Nunally. One of ordinary skill in the art would 
be motivated to make this combination in order to update the index information for video 
and audio files in view of Nunally (col. 10, lines 37-62), as doing so would give the 
added benefit of providing an integrated video and audio recording device with 
advanced information management capabilities as taught by Nunally (col. 1, lines 37- 
39). 

As per claim 38, Pan teaches an apparatus for storing video data and 
associated text data (i.e. textual data stream, col. 11, lines 16-34), said apparatus 
comprising of: 

media file generation means for storing the video data (i.e. video and audio data 
is stored on a hard disk, col. 4, lines 58-65) and associated text data (i.e. textual data 
stream, col. 11, lines 16-34), as one or more data samples (i.e. a sequence of video and 
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audio frames, col. 3, lines 20-27; event e1, ... event e6; event f1 .... event f5; See Fig. 
2), in one or more media files in accordance with a first file format (i.e. Data stream #1, 
Data stream #2 in Fig. 2; a digital file, col. 1, line 59 to col. 2, line 5), each media file 
configured for use by a media player application (i.e. a media file that can be played 
back through an application such as Microsoft Windows Media Player, col. 3, lines 20- 
27) in playing the video data (i.e. a "data stream" is a sequence of video and audio 
frames read from a media file that can be played back through an application such as 
Microsoft Windows Media Player, col. 3, lines 20-27); 

index file generation means for storing in an index file in accordance with a 
second file format (i.e. a data set is maintained in an index file, col. 2, lines 11-26), at 
least an offset value (i.e. positional information, col. 1, line 59 to col. 2, line 5) for each 
of the data samples representing a location of each of the one or more data samples in 
a corresponding one of the media files (i.e. positional information indicating where data 
associated with the events are located in a corresponding storage medium such as a 
digital file or where the data is located within the data stream, col. 1, line 59 to col. 2, 
line 5); and 

image information adding means (i.e. informational data set is generated and 
stored in an index file, Abstract) for adding additional information (i.e. data pointer, time, 
See Fig. 2) interspersed throughout each of the media files (i.e. correlating data among 
multiple data streams based on a use of time-stamps and related positional information, 
col. 1, lines 46-58; See Fig. 2), the media file comprising the additional information 
being readable by the media player application corresponding at least to the first file 
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format (i.e. a media file that can be played back through an application such as 
Microsoft Windows Media Player, col. 3, lines 20-27), the additional information 
comprising at least a timestamp (i.e. the time-stamps are used as an index to locate 
where the data associated with the corresponding event is stored in the second data 
stream, col. 2, lines 27-40) for one or more data samples (i.e. event e1, ... event e6; 
event f1 .... event f5; See Fig. 2), each of the timestamps (i.e. t1, t3, t10, t6, t8, t11, t2, 
t4, t5, t7, t9; See Fig. 2) indicating a capture time of an associated data sample (i.e. 
During a presentation of information at slide display device 580, an audio-video 
recording unit 585 captures details of, for example, a corresponding slide presentation 
including a lecturer delivering a speech. A data stream generated by the audio-video 
recording unit 585 is captured for storage in a data file at data processing unit 560, col. 
10, lines 39-51). 

It should be noted that Pan teaches the additional information of the media file is 
used in constructing the index file as shown in Fig. 2 (i.e. a data set is maintained in an 
index file, col. 2, lines 11-26; The resulting event-data pointer record 152, 
(E.sub.i,j,P.sub.i,j,T.sub.i,j), is then stored in a data stream index file #1 235. There is 
usually one index file for each data stream, Pan, col. 4, lines 34-39), wherein the data 
set of Pan comprises Event E (data sample of the claimed limitation), Pointer P 
(additional information of the claimed limitation), Timestamp T (additional information of 
the claimed limitation). 

The index file of Pan further comprises the offset values (i.e. positional 
information, col. 1, line 59 to col. 2, line 5) representing the location of each of the data 
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samples in the media files (i.e. positional information indicating where data associated 
with the events are located in a corresponding storage medium such as a digital file or 
where the data is located within the data stream, col. 1, line 59 to col. 2, line 5). 
Pan does not explicitly teach: 

the additional information of the media files being used in reconstructing the 
index file upon corruption of the index file, the reconstructed index file comprising the 
offset values representing the location of each of the data samples in the media files, 
wherein the reconstructed index file replaces the index file associated with the one or 
more media files. 

Lyle teaches: 

the additional information (i.e. additional information within a space map page in 
a LOB table space, col. 4, line 60 to col. 5, line 6) of the media files (i.e. LOB, Large 
objects, such as image data, col. 2, lines 27-37) being used in reconstructing the index 
file (i.e. to recover an index on an auxiliary table by reading only the LOB low-level 
space map pages, col. 4, line 60 to col. 5, line 6) upon corruption of the index file (i.e. 
the index could be corrupted, col. 2, lines 38-44), the reconstructed index file comprising 
the offset values (i.e. the index recovery system 124 indicates for each page in the 
space map page whether that page is the first page allocated to a LOB, col. 4, line 60 to 
col. 5, line 6) representing the location of each of the data samples in the media files 
(i.e. The index recovery system 124 of the present invention includes additional 
information within a space map page in a LOB table space. In addition to recording 
whether a page is allocated or deallocated, the index recovery system 124 indicates for 
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each page in the space map page whether that page is the first page allocated to a 
LOB. Storing this information with a LOB low-level space map page enables the index 
recovery system 124 to recover an index on an auxiliary table by reading only the LOB 
low-level space map pages, instead of all pages in the LOB table space, col. 4, line 60 
to col. 5, line 6). 

It should be noted that the claimed invention discloses using additional 
information such as image information to reconstruct the index file (said image 
information allows for reconstruction of said index file upon corruption thereof, See 
Application, [0017]). 

Similarly, Lyle teaches the using of additional information such as image 
information in order to rebuild the index. 

Thus, it would have been obvious to one of ordinary skill of the art having the 
teaching of Pan and Lyle at the time the invention was made to modify the system of 
Pan to include the limitations as taught by Lyle. One of ordinary skill in the art would be 
motivated to make this combination in order to recover an index on an auxiliary table in 
view of Lyle (col. 4, line 60 to col. 5, line 6), as doing so would give the added benefit of 
enabling the index recovery system 124 to recover an index on an auxiliary table by 
reading only the LOB low-level space map pages, instead of all pages in the LOB table 
space as taught by Lyle (col. 4, line 60 to col. 5, line 6). 

Pan, Lyle do not specifically teach the reconstructed index file replaces the index 
file associated with the one or more media files. 
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Nunally teaches the step of updating the index information for video and audio 
files (i.e. the sequence is added to a positive result list (step 152) and the index 
information for the file is updated to indicate detection of the event (step 154). That is, 
the event-related data shown at 104 in FIG. 5 is updated to indicate detection of the 
event, as well as the confidence factor applicable to the event detection decision, col. 
10, lines 37-62; FIG. 5 illustrates a format in which compressed video and audio data 
are stored on one or more of the hard disk drives of the analysis/storage device. As 
seen from FIG. 5, the data stored on the hard drives includes compressed video and 
audio data files indicated generally at 92 and index data indicated generally at 94, col. 
8, line 66 to col. 9, line 14). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally at the time the invention was made to modify the system of Pan, 
Lyle to include the limitations as taught by Nunally. One of ordinary skill in the art would 
be motivated to make this combination in order to update the index information for video 
and audio files in view of Nunally (col. 10, lines 37-62), as doing so would give the 
added benefit of providing an integrated video and audio recording device with 
advanced information management capabilities as taught by Nunally (col. 1, lines 37- 
39). 

As per claim 40, Pan teaches a computer product comprising a computer 
readable medium having recorded thereon a computer program for storing data (i.e. 
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video and audio data is stored on a hard disk, col. 4, lines 58-65), said program 
comprising: 

code for storing data (i.e. video and audio data is stored on a hard disk, col. 4, 
lines 58-65), as one or more data samples (i.e. a sequence of video and audio frames, 
col. 3, lines 20-27; event e1, ... event e6; event f1 .... event f5; See Fig. 2), in one or 
more media files configured for use by a media player application (i.e. a media file that 
can be played back through an application such as Microsoft Windows Media Player, 
col. 3, lines 20-27) in playing the data samples (i.e. a "data stream" is a sequence of 
video and audio frames read from a media file that can be played back through an 
application such as Microsoft Windows Media Player, col. 3, lines 20-27); and 

code for storing in an index file associated with one or more of the media files 
(i.e. a data set is maintained in an index file, col. 2, lines 11-26), at least an offset value 
(i.e. positional information, col. 1, line 59 to col. 2, line 5) for each of the data samples 
representing a location (i.e. positional information indicating where data associated with 
the events are located in a corresponding storage medium such as a digital file or where 
the data is located within the data stream, col. 1, line 59 to col. 2, line 5) of each of the 
data samples in a corresponding one of the media files, each of the media files further 
comprising additional information (i.e. data pointer, time, See Fig. 2) interspersed 
throughout that media file (i.e. correlating data among multiple data streams based on a 
use of time-stamps and related positional information, col. 1, lines 46-58; See Fig. 2), 
the additional information comprising at least a timestamp (i.e. the time-stamps are 
used as an index to locate where the data associated with the corresponding event is 
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stored in the second data stream, col. 2, lines 27-40) for each of the data samples (i.e. 
event e1, ... event e6; event f1 .... event f5; See Fig. 2), each of the timestamps (i.e. t1, 
t3, t10, t6, t8, t11, t2, t4, t5, t7, t9; See Fig. 2) indicating a capture time of an associated 
data sample (i.e. During a presentation of information at slide display device 580, an 
audio-video recording unit 585 captures details of, for example, a corresponding slide 
presentation including a lecturer delivering a speech. A data stream generated by the 
audio-video recording unit 585 is captured for storage in a data file at data processing 
unit 560, col. 10, lines 39-51). 

It should be noted that Pan teaches the additional information of the media file is 
used in constructing the index file as shown in Fig. 2 (i.e. a data set is maintained in an 
index file, col. 2, lines 11-26; The resulting event-data pointer record 152, 
(E.sub.i,j,P.sub.i,j,T.sub.i,j), is then stored in a data stream index file #1 235. There is 
usually one index file for each data stream, Pan, col. 4, lines 34-39), wherein the data 
set of Pan comprises Event E (data sample of the claimed limitation), Pointer P 
(additional information of the claimed limitation), Timestamp T (additional information of 
the claimed limitation). 

The index file of Pan further comprises the offset values (i.e. positional 
information, col. 1, line 59 to col. 2, line 5) representing the location of each of the data 
samples in the media files (i.e. positional information indicating where data associated 
with the events are located in a corresponding storage medium such as a digital file or 
where the data is located within the data stream, col. 1, line 59 to col. 2, line 5). 

Pan does not explicitly teach: 
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the additional information of the media files being used in reconstructing the 
index file upon corruption of the index file, the reconstructed index file comprising the 
offset values representing the location of each of the data samples in the media files, 
wherein the reconstructed index file replaces the index file associated with the one or 
more media files. 

Lyle teaches: 

the additional information (i.e. additional information within a space map page in 
a LOB table space, col. 4, line 60 to col. 5, line 6) of the media files (i.e. LOB, Large 
objects, such as image data, col. 2, lines 27-37) being used in reconstructing the index 
file (i.e. to recover an index on an auxiliary table by reading only the LOB low-level 
space map pages, col. 4, line 60 to col. 5, line 6) upon corruption of the index file (i.e. 
the index could be corrupted, col. 2, lines 38-44), the reconstructed index file comprising 
the offset values (i.e. the index recovery system 124 indicates for each page in the 
space map page whether that page is the first page allocated to a LOB, col. 4, line 60 to 
col. 5, line 6) representing the location of each of the data samples in the media files 
(i.e. The index recovery system 124 of the present invention includes additional 
information within a space map page in a LOB table space. In addition to recording 
whether a page is allocated or deallocated, the index recovery system 124 indicates for 
each page in the space map page whether that page is the first page allocated to a 
LOB. Storing this information with a LOB low-level space map page enables the index 
recovery system 124 to recover an index on an auxiliary table by reading only the LOB 
low-level space map pages, instead of all pages in the LOB table space, col. 4, line 60 



Application/Control Number: 10/748,334 Page 27 

Art Unit: 2169 

to col. 5, line 6). 

It should be noted that the claimed invention discloses using additional 
information such as image information to reconstruct the index file (i.e. said image 
information allows for reconstruction of said index file upon corruption thereof, See 
Application, [0017]). Similarly, Lyle teaches the using of additional information such as 
image information to rebuild the index. 

Thus, it would have been obvious to one of ordinary skill of the art having the 
teaching of Pan and Lyle at the time the invention was made to modify the system of 
Pan to include the limitations as taught by Lyle. One of ordinary skill in the art would be 
motivated to make this combination in order to recover an index on an auxiliary table in 
view of Lyle (col. 4, line 60 to col. 5, line 6), as doing so would give the added benefit of 
enabling the index recovery system 124 to recover an index on an auxiliary table by 
reading only the LOB low-level space map pages, instead of all pages in the LOB table 
space as taught by Lyle (col. 4, line 60 to col. 5, line 6). 

Pan, Lyle do not fairly teach the reconstructed index file replaces the index file 
associated with the one or more media files. 

Nunally teaches the step of updating the index information for video and audio 
files (i.e. the sequence is added to a positive result list (step 152) and the index 
information for the file is updated to indicate detection of the event (step 154). That is, 
the event-related data shown at 104 in FIG. 5 is updated to indicate detection of the 
event, as well as the confidence factor applicable to the event detection decision, col. 
10, lines 37-62; FIG. 5 illustrates a format in which compressed video and audio data 
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are stored on one or more of the hard disk drives of the analysis/storage device. As 
seen from FIG. 5, the data stored on the hard drives includes compressed video and 
audio data files indicated generally at 92 and index data indicated generally at 94, col. 
8, line 66 to col. 9, line 14). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally at the time the invention was made to modify the system of Pan, 
Lyle to include the limitations as taught by Nunally. One of ordinary skill in the art would 
be motivated to make this combination in order to update the index information for video 
and audio files in view of Nunally (col. 1 0, lines 37-62), as doing so would give the 
added benefit of providing an integrated video and audio recording device with 
advanced information management capabilities as taught by Nunally (col. 1, lines 37- 
39). 

As per claim 41, a computer product comprising a computer readable medium 
having recorded thereon a computer program for storing video (i.e. video and audio 
data is stored on a hard disk, col. 4, lines 58-65) and associated text data (i.e. textual 
data stream, col. 11, lines 16-34), said program comprising: 

code for storing the video data (i.e. video and audio data is stored on a hard disk, 
col. 4, lines 58-65) associated text data (i.e. textual data stream, col. 11, lines 16-34), as 
one or more data samples (i.e. a sequence of video and audio frames, col. 3, lines 20- 
27; event e1, ... event e6; event f1 .... event f5; See Fig. 2), in one or more media files 
in accordance with a first file format (i.e. Data stream #1, Data stream #2 in Fig. 2; a 



Application/Control Number: 10/748,334 Page 29 

Art Unit: 2169 

digital file, col. 1, line 59 to col. 2, line 5), each media file being configured for use by a 
media player application (i.e. a media file that can be played back through an 
application such as Microsoft Windows Media Player, col. 3, lines 20-27) in playing the 
video data (i.e. a "data stream" is a sequence of video and audio frames read from a 
media file that can be played back through an application such as Microsoft Windows 
Media Player, col. 3, lines 20-27); 

code for storing, in an index file in accordance with a second file format (i.e. a 
data set is maintained in an index file, col. 2, lines 11-26), at least an offset value (i.e. 
positional information, col. 1, line 59 to col. 2, line 5) for each of the data samples 
representing a location of each of the one or more data samples in a corresponding one 
of the media files (i.e. positional information indicating where data associated with the 
events are located in a corresponding storage medium such as a digital file or where the 
data is located within the data stream, col. 1, line 59 to col. 2, line 5); and 

code for adding (i.e. informational data set is generated and stored in an index 
file, Abstract) additional information (i.e. data pointer, time, See Fig. 2) interspersed 
throughout each of the media files (i.e. correlating data among multiple data streams 
based on a use of time-stamps and related positional information, col. 1, lines 46-58; 
See Fig. 2), the media file including the additional information being readable by the 
media player application corresponding at least to the first file format (i.e. a media file 
that can be played back through an application such as Microsoft Windows Media 
Player, col. 3, lines 20-27), the additional information comprising at least a timestamp 
(i.e. the time-stamps are used as an index to locate where the data associated with the 
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corresponding event is stored in the second data stream, col. 2, lines 27-40) for one or 
more data samples (i.e. event e1, ... event e6; event f1 .... event f5; See Fig. 2), each 
of the timestamps (i.e. t1, t3, t10, t6, t8, t11, t2, t4, t5, t7, t9; See Fig. 2) indicating a 
capture time of an associated data sample (i.e. During a presentation of information at 
slide display device 580, an audio-video recording unit 585 captures details of, for 
example, a corresponding slide presentation including a lecturer delivering a speech. A 
data stream generated by the audio-video recording unit 585 is captured for storage in a 
data file at data processing unit 560, col. 10, lines 39-51). 

It should be noted that Pan teaches the additional information of the media file is 
used in constructing the index file as shown in Fig. 2 (i.e. a data set is maintained in an 
index file, col. 2, lines 11-26; The resulting event-data pointer record 152, 
(E.sub.i,j,P.sub.i,j,T.sub.i,j), is then stored in a data stream index file #1 235. There is 
usually one index file for each data stream, Pan, col. 4, lines 34-39), wherein the data 
set of Pan comprises Event E (data sample of the claimed limitation), Pointer P 
(additional information of the claimed limitation), Timestamp T (additional information of 
the claimed limitation). 

The index file of Pan further comprises the offset values (i.e. positional 
information, col. 1, line 59 to col. 2, line 5) representing the location of each of the data 
samples in the media files (i.e. positional information indicating where data associated 
with the events are located in a corresponding storage medium such as a digital file or 
where the data is located within the data stream, col. 1, line 59 to col. 2, line 5). 

Pan does not explicitly teach: 



Application/Control Number: 10/748,334 Page 31 

Art Unit: 2169 

the additional information of the media files being used in reconstructing the 
index file upon corruption of the index file, the reconstructed index file comprising the 
offset values representing the location of each of the data samples in the media files, 
wherein the reconstructed index file replaces the index file associated with the one or 
more media files. 

Lyle teaches: 

the additional information (i.e. additional information within a space map page in 
a LOB table space, col. 4, line 60 to col. 5, line 6) of the media files (i.e. LOB, Large 
objects, such as image data, col. 2, lines 27-37) being used in reconstructing the index 
file (i.e. to recover an index on an auxiliary table by reading only the LOB low-level 
space map pages, col. 4, line 60 to col. 5, line 6) upon corruption of the index file (i.e. 
the index could be corrupted, col. 2, lines 38-44), the reconstructed index file comprising 
the offset values (i.e. the index recovery system 124 indicates for each page in the 
space map page whether that page is the first page allocated to a LOB, col. 4, line 60 to 
col. 5, line 6) representing the location of each of the data samples in the media files 
(i.e. The index recovery system 124 of the present invention includes additional 
information within a space map page in a LOB table space. In addition to recording 
whether a page is allocated or deallocated, the index recovery system 124 indicates for 
each page in the space map page whether that page is the first page allocated to a 
LOB. Storing this information with a LOB low-level space map page enables the index 
recovery system 124 to recover an index on an auxiliary table by reading only the LOB 
low-level space map pages, instead of all pages in the LOB table space, col. 4, line 60 
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to col. 5, line 6). 

It should be noted that the Applicants invention discloses using additional 
information such as image information in order to reconstruct the index file (i.e. said 
image information allows for reconstruction of said index file upon corruption thereof, 
See Application, [0017]). 

Similarly, Lyle teaches the using of additional information such as image 
information in order to rebuild the index. 

Thus, it would have been obvious to one of ordinary skill of the art having the 
teaching of Pan and Lyle at the time the invention was made to modify the system of 
Pan to include the limitations as taught by Lyle. One of ordinary skill in the art would be 
motivated to make this combination in order to recover an index on an auxiliary table in 
view of Lyle (col. 4, line 60 to col. 5, line 6), as doing so would give the added benefit of 
enabling the index recovery system 124 to recover an index on an auxiliary table by 
reading only the LOB low-level space map pages, instead of all pages in the LOB table 
space as taught by Lyle (col. 4, line 60 to col. 5, line 6). 

Pan, Lyle do not fairly teach the reconstructed index file replaces the index file 
associated with the one or more media files. 

Nunally teaches the step of updating the index information for video and audio 
files (i.e. the sequence is added to a positive result list (step 152) and the index 
information for the file is updated to indicate detection of the event (step 154). That is, 
the event-related data shown at 104 in FIG. 5 is updated to indicate detection of the 
event, as well as the confidence factor applicable to the event detection decision, col. 
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10, lines 37-62; FIG. 5 illustrates a format in which compressed video and audio data 
are stored on one or more of the hard disk drives of the analysis/storage device. As 
seen from FIG. 5, the data stored on the hard drives includes compressed video and 
audio data files indicated generally at 92 and index data indicated generally at 94, col. 
8, line 66 to col. 9, line 14). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally at the time the invention was made to modify the system of Pan, 
Lyle to include the limitations as taught by Nunally. One of ordinary skill in the art would 
be motivated to make this combination in order to update the index information for video 
and audio files in view of Nunally (col. 10, lines 37-62), as doing so would give the 
added benefit of providing an integrated video and audio recording device with 
advanced information management capabilities as taught by Nunally (col. 1, lines 37- 
39). 

As to claims 2, 23, Lyle teaches the additional information is used exclusively 
for reconstruction of the index file (i.e. to recover an index on an auxiliary table by 
reading only the LOB low-level space map pages, col. 4, line 60 to col. 5, line 6). 

As to claims 7, 28, Pan teaches the additional information comprises a 
resolution of an associated sample (i.e. The consequences of latency depend upon the 
application and the desired time resolution at which queries are being made, col. 12, 
lines 46-55). 
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As to claims 8, 29, Pan teaches the information of the index file comprises 
frame rate variation information (i.e. video rate of 30 frames per second, col. 11, line 58 
to col. 12, line 2). 

As to claims 9, 30, Pan teaches the additional information is stored as one or 
more dedicated samples of the media file (i.e. FIG. 3 is a graph of sample trigger events 
as recorded overtime according to sample data in the table of FIG. 2. According to the 
principles of the present invention, events in one data file are indexed to events in a 
second data file based on time-stamps and corresponding data pointers, col. 7, lines 
10-14). 

As to claims 10, 31, Nunally teaches the media file is configured in accordance 
with the Microsoft AVI file format (i.e. Both the video data and the audio data are stored 
in an audio/video interleave (AVI) format, col. 8, line 66 to col. 9, line 14). 

As per claim 12, Pan teaches the data is video data (i.e. video frames, col. 11, 
lines 45-52). 

As per claim 13, Pan teaches the data is text data (i.e. textual data stream, col. 
11, lines 16-34). 
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As per claim 14, Pan teaches the data is video data and associated text data 
(i.e. closed captions for video or audio content, col. 11, lines 16-23). 

As to claims 15, 33, Pan teaches the video and associated text data are 
captured for security purposes (i.e. The principles of the present invention have 
applications beyond note taking, text entry, or a close captioning system. One example 
is a video surveillance system for an automobile parking lot. This example system uses 
multiple video cameras. One set of cameras record the automobiles and associated 
license plates at the entrance and exits. Another set of cameras record the parking 
area. In current video surveillance systems, cameras continuously record video on tape. 
This results in a waste of video recording media. A more efficient system would record, 
for example, only for a few seconds after a motion detector indicates when a car has 
entered or left the garage or parking space. This type of sporadic recording renders it 
difficult to determine where an event is recorded on a videotape. Even a time-stamped 
videotape must be searched for the appropriate time match, col. 13, line 62 to col. 14, 
line 9). 

As per claim 17, Nunally teaches a plurality of copies of a corresponding text 
string are included in each text sample of the media file (i.e. sound, image orclosed- 
captioning analysis may be applied to a previously-recorded signal reproduced from the 
storage device 326 and routed to the analysis circuit 324 via a signal path 330, col. 16, 
line 32 to col. 17, line 2). 
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As per claim 18, Nunally teaches a first copy of the text string (i.e. sound, image 
or closed-captioning analysis may be applied to a previously-recorded signal 
reproduced from the storage device 326 and routed to the analysis circuit 324 via a 
signal path 330, col. 16, line 32 to col. 17, line 2) is configured in accordance with the 
AVI file format (i.e. Both the video data and the audio data are stored in an audio/video 
interleave (AVI) format, col. 8, line 66 to col. 9, line 14). 

As to claims 43-46, Pan teaches the additional information comprises at least a 
timestamp (i.e. the time-stamps are used as an index to locate where the data 
associated with the corresponding event is stored in the second data stream, col. 2, 
lines 27-40) for one or more of the data samples adjacent to the additional information 
(i.e. data pointer, time, See Fig. 2). 

Claims 11, 16, 19, 32 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Pan et al. (US Patent No. 6,993,246), in view of Lyle (US Patent 
No. 6,470,359), in view of Nunally et al. (US Patent No. 6,035,341), and further in 
view of Otsuka et al. (US Patent No. 6,065,010). 

As to claims 11, 32, Pan, Lyle, Nunally do not specifically teach the index file is 
configured in accordance with the Apple QuickTime file format. 

Otsuka teaches the index file is configured in accordance with the Apple 
QuickTime file format (i.e. the data of the scenes can be recorded in a number of 
different formats including: the Motion Picture Experts Group (MPEG) format; the 
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Quicktime format; or the Joint Photographic Experts Group (JPEG) format, col. 5, line 
49 to col. 6, line 2). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally, Otsuka at the time the invention was made to modify the system 
of Pan, Lyle, Nunally to include the limitations as taught by Otsuka. One of ordinary skill 
in the art would be motivated to make this combination in order to define virtual files, 
each virtual file is associated with a separate individual data of a group of data within a 
physical file in view of Otsuka (col. 6, lines 3-15), as doing so would give the added 
benefit of the data of accessing the virtual file rapidly and efficiently accessed without 
the need to search through the physical file for it as taught by Otsuka (col. 6, lines 3-15). 

As per claim 16, Pan, Lyle, Nunally do not specifically teach each video sample 
is a separate JPEG file. 

Otsuka teaches this limitation (i.e. the data of the scenes can be recorded in a 
number of different formats including: the Motion Picture Experts Group (MPEG) format; 
the Quicktime format; or the Joint Photographic Experts Group (JPEG) format, col. 5, 
line 49 to col. 6, line 2). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally, Otsuka at the time the invention was made to modify the system 
of Pan, Lyle, Nunally to include the limitations as taught by Otsuka. One of ordinary skill 
in the art would be motivated to make this combination in order to define virtual files, 
each virtual file is associated with a separate individual data of a group of data within a 
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physical file in view of Otsuka (col. 6, lines 3-1 5), as doing so would give the added 
benefit of accessing the data of the virtual file rapidly and efficiently accessed without 
the need to search through the physical file for it as taught by Otsuka (col. 6, lines 3-15). 

As per claim 19, Pan, Lyle, Nunally do not specifically teach a second copy of 
the text string is configured in accordance with the QuickTime file format. 

Otsuka teaches this limitation (i.e. the data of the scenes can be recorded in a 
number of different formats including: the Motion Picture Experts Group (MPEG) format; 
the Quicktime format; or the Joint Photographic Experts Group (JPEG) format, col. 5, 
line 49 to col. 6, line 2). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally, Otsuka at the time the invention was made to modify the system 
of Pan, Lyle, Nunally to include the limitations as taught by Otsuka. One of ordinary skill 
in the art would be motivated to make this combination in order to define virtual files, 
each virtual file is associated with a separate individual data of a group of data within a 
physical file in view of Otsuka (col. 6, lines 3-1 5), as doing so would give the added 
benefit of accessing the data of the virtual file rapidly and efficiently without the need to 
search through the physical file for it as taught by Otsuka (col. 6, lines 3-15). 

Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pan 
et al. (US Patent No. 6,993,246), in view of Lyle (US Patent No. 6,470,359), in view 
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of Nunally et al. (US Patent No. 6,035,341), and further in view of Gutfreund et al. 
(US Patent No. 6,665,835). 

As per claim 20, Pan, Lyle, Nunally do not specifically teach the step of inserting 
one or more empty samples into the media file to compensate for any missed samples. 

Gutfreund teaches this limitation (i.e. A reference for the timer concerning where 
in the video stream the event occurs is maintained by creating a series of 
predetermined time stamps in the video stream. That is, a time stamp is created every 
second with no component or event associated therewith. The granularity of such 
predetermined time stamps is determined according to the specific application 
requirements, col. 8, line 61 to col. 8, line 14). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally, Gutfreund at the time the invention was made to modify the 
system of Pan, Lyle, Nunally to include the limitations as taught by Gutfreund. One of 
ordinary skill in the art would be motivated to make this combination in order to create a 
series of predetermined time stamp in the video stream in view of Gutfreund (col. 8, line 
61 to col. 8, line 14), as doing so would give the added benefit of providing a real-time 
multimedia journaling application that is editable by a user to add other synchronized 
events as taught by Gutfreund (col. 2, lines 34-36). 

Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pan 
et al. (US Patent No. 6,993,246), in view of Lyle (US Patent No. 6,470,359), in view 
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of Nunally et al. (US Patent No. 6,035,341), and further in view of Abbott et al. (US 
Patent No. 6,654,933). 

As per claim 21, Pan, Lyle, Nunally do not teach the index file contains a track 
referencing at least the media file. 

Abbott teaches this limitation (i.e. the audio and video tracks of a movie, col. 5, 
lines 3-27). 

It would have been obvious to one of ordinary skill of the art having the teaching 
of Pan, Lyle, Nunally, Abbott at the time the invention was made to modify the system of 
Pan, Lyle, Nunally to include the limitations as taught by Abbott. One of ordinary skill in 
the art would be motivated to make this combination in order to encode the audio and 
video tracks and store in a file in view of Abbott (col. 4, lines 3-27), as doing so would 
give the added benefit of synchronization method including constructing a base atom 
index file that contains base atom index boundaries as taught by Abbott (col. 2, lines 45- 
54). 

Response to Arguments 

Applicant's arguments filed 05/20/2008 have been fully considered but they are 
not persuasive. 

a. Pan does teach storing data, as one or more data samples, in one or 
more media files configured for use by a media player application in playing the 
data samples, and each of the media files further comprising additional 
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information interspersed throughout that media file, the additional information 
comprising at least a timestamp for one or more of the data samples. 

First, the limitation data samples - as described in the instant specification, 
paragraph [0019], comprises video data and text data. 

The independent claim should be amended to specify " data samples including 
video data and text data ". 

Second, as described in the specification, one or more data samples is (are) 
stored in only one media file as shown in Figs 4a, b, AVI.TM. File. However, the claim 
language recites storing one or more data samples in one or more media files . 

Therefore, the plurality of sets (event, data ptr, time), as shown in Fig. 2 of Pan, 
are stored data stream #1 , #2 as one or more media files limitation, wherein each set 
(event, data ptr, time) corresponds to a data sample limitation. 

The independent claim should be amended to clarify that one or more data 
samples store in only one media file . 

It is noted that although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

Specifically, Pan reads on the claimed limitations as follows: 

The step of storing data limitation equates to "The resulting event-data pointer 
record is then stored in a data stream index file " of Pan (See col. 4, lines 34-49). 
Furthermore, Pan discloses the video and audio data is stored on a hard disk in order to 
play back (See col. 4, lines 58-65). 
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data samples of the claimed limitation equates to a sequence of video and audio 
frames (col. 3, lines 20-27), or a set of events (col. 3, line 62 to col. 4, line 4; See Fig. 
2). 

the video and associated text data limitation is taught by Pan as in col. 1, 

lines 45-58 (i.e. The present invention is a novel apparatus and method generally 
directed towards correlating data among multiple data streams based on a use of time- 
stamps and related positional information. One potential application is a closed 
captioned system in which text is generated for display along with corresponding video 
data . For example, closed captions are textual transcriptions of an audio track of a 
television show that are displayed on a television screen at approximately the same 
time as corresponding video data. According to the principles of the present invention, 
the audio track or audio data stream can be correlated with the textual data stream to 
support synchronization of such streams during playback, col. 1, lines 45-58). 

media files of the claimed limitation equates to Data stream #1, Data stream #2 
in Fig. 2 or a digital file of Pan, See col. 1 , line 59 to col. 2, line 5 (i.e. positional 
information indicating where data associated with the events are located in a 
corresponding storage medium such as a digital file or where the data is located within 
the data stream, col. 1, line 59 to col. 2, line 5). 

a media player application limitation equates to Microsoft Windows Media 
Player of Pan in col. 3, lines 20-27 (i.e. a "data stream" is a sequence of video and 
audio frames read from a media file that can be played back through an application 
such as Microsoft Windows Media Player, See Pan, col. 3, lines 20-27). 
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additional information limitation equates to Ptr, time of Pan in Fig. 2. 

It should be noted that the time T of Pan is a unique time stamp T to each event, 
See col. 4, lines 34-39 (i.e. Sequences of event-data pointer pairs form the inputs to 
event-data pointer logger 250. The logger assigns a unique time stamp Tij to each pair 
(E.i,j,P.i,j). The resulting event-data pointer record 152, (Ei,i,P.i,i, Tij), is then stored in a 
data stream index file #1 235 . There is usually one index file for each data stream, Pan, 
col. 4, lines 34-39). 

additional information interspersed throughout that media file limitation as 
shown in Fig. 2 of Pan, e.g. t2, t4, t5, t7, t9 are interspersed throughout in data stream 
#2 (See Fig. 2). 

b. Lyle and Nunally does teach the claimed features of "the reconstructed 
file replaces index file associated with the one or more media files". 

Lyle discloses an index recovery system and method including the step of 
rebuilding the index, this corresponds to the reconstructed file index file limitation (i.e. 
when a RDBMS stores LOBs, an index may be used to access the LOBs efficiently. 
However, when computer systems fail, the index could be corrupted or destroyed. In 
this case , recovery of the index, which involves rebuilding the index, can be very time 
consuming because each page that was allocated to storing each LOB must be read, 
col. 2, lines 38-44, Lyle). 

The large objects (LOBs) of Lyle with which the index associated are image data, 
therefore, the image data of Lyle equate to media file limitation (i.e. a RDBMS stored 
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simple data, such as numeric and text data. In a traditional RDBMS, the underlying 
storage management has been optimized for simple data. More specifically, the size of 
a record is limited by the size of a page, which is a fixed number (e.g., 4K) defined by a 
computer developer. This restriction in turn poses a limitation on the length of columns 
of a table. To alleviate such a restriction, most computer developers today support a 
new built-in data type for storing large objects (LOBs). Large objects, such as image 
data, typically take up a great deal of storage space, col. 2, lines 27-37, Lyle). 

As discussed in the office action, Pan teaches the additional information of the 
media file is used in constructing the index file as showed in Fig. 2 (i.e. a data set is 
maintained in an index file, col. 2, lines 11-26; The resulting event-data pointer record 
152, (E.sub.i,j,P.sub.i,j,T.sub.i,j), is then stored in a data stream index file #1 235. There 
is usually one index file for each data stream, Pan, col. 4, lines 34-39, Pan). 

Pan does not fairly teach the step of " reconstructing ". 

Lyle teaches the step of reconstructed infix file limitation (i.e. the index 
recovery system 124 indicates for each page in the space map page whether that page 
is the first page allocated to a LOB, col. 4, line 60 to col. 5, line 6, Lyle), the index file of 
Lyle is associated with image data - media file limitation (i.e. Large objects, such as 
image data, typically take up a great deal of storage space, col. 2, lines 27-37, Lyle). 

Lyle however does not teach the media file as video data and text data. 

Nunally teaches a media file storing video data and text data sample, and an 
index file associated with the media file (i.e. Alternatively, as suggested above, the 
results of closed captioning analysis could be utilized to index particular portions of 
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interest in a TV signal stream that has previously been selected for recording, with the 
indexing taking place as the signal is received. Moreover, as suggested by previous 
discussion, sound, image or closed-captioning analysis may be applied to a previously- 
recorded signal reproduced from the storage device 326 and routed to the analysis 
circuit 324 via a signal path 330, col. 16, line 32 to col. 17, line 2, Nunally), wherein the 
video data incoming from the cameras in the surveillance system (col. 4, lines 1-17, 
Nunally) as the applicants system invention. 

In response to applicant's argument that the cited references teach away from 
the claimed features, the Examiner disagrees. The present invention is directed to 
method for storing video and text data samples. Similarly, Pan is directed to method for 
generating audio video data set and storing in an index file for each detected trigger 
event. Lyle is directed to method for storing image data using index techniques for 
faster retrieval. Analogously, Nunally is directed to a method of storing and retrieving 
audio data. The three references disclose the same field as the claimed invention as 
video image processing. Therefore, it would have been obvious to one of ordinary skill 
of the art having the teaching of Pan, Lyle, Nunally at the time the invention was made 
to modify the system of Pan, Lyle to include the limitations as taught by Nunally. One of 
ordinary skill in the art would be motivated to make this combination in order to update 
the index information for video and audio files in view of Nunally (col. 10, lines 37-62), 
as doing so would give the added benefit of providing an integrated video and audio 
recording device with advanced information management capabilities as taught by 
Nunally (col. 1, lines 37-39). 
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The Examiner notes the fact that applicant has recognized another advantage 
which would flow naturally from following the suggestion of the prior art cannot be the 
basis for patentability when the differences would otherwise be obvious. See Ex parte 
Obiaya, 227 USPQ 58, 60 (Bd. Pat. App. & Inter. 1985). 

It is further noted that Applicant's arguments against the references individually, 
Applicant is reminded that one cannot show nonobviousness by attacking references 
individually where the rejections are based on combinations of references. See In re 
Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 
231 USPQ 375 (Fed. Cir. 1986). It is a well settled rule that a reference must be 
considered not only for what it expressly teaches but also for what it fairly suggests. See 
In re Burckel, 592 F.2d 1175,201 USPQ 67 (CCPA 1979) and In re Lamberti, 545 F.2d 
747, 192 USPQ 278 (CCPA 1976) as well as In re Bode, 550 F.2d 656, 193 USPQ 12 
(CCPA 1977) which indicates such fair suggestions to unpreferred embodiments must 
be considered even if they were not illustrated. Additionally, it is an equally well settled 
rule that what a reference can be said to fairly suggest relates to the concepts fairly 
contained therein, and is not limited by the specific structure chosen to illustrate such 
concepts. See In re Bascom, 230 F.2d 612, 109 USPQ 98 (CCPA 1956). 

According to KSR, all claimed elements were known in the prior art and one 
skilled in the art could have combined the elements as claimed by known methods with 
no change in their respective functions, and the combination would have yielded 
predictable results to a skilled artisan at the time the invention was made. 
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c. As to claims 43-46, Pan teaches the additional information comprises at least 
a timestamp (i.e. the time-stamps are used as an index to locate where the data 
associated with the corresponding event is stored in the second data stream, col. 2, 
lines 27-40) for one or more of the data samples adjacent to the additional information 
(i.e. data pointer, time, See Fig. 2). 

Accordingly, the claimed invention as represented in the claims does not represent a 
patentable over the art of record. 



Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 

MONTHS from the mailing date of this action. In the event a first reply is filed within 

TWO MONTHS of the mailing date of this final action and the advisory action is not 

mailed until after the end of the THREE-MONTH shortened statutory period, then the 

shortened statutory period will expire on the date the advisory action is mailed, and any 

extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 

the advisory action. In no event, however, will the statutory period for reply expire later 

than SIX MONTHS from the mailing date of this final action. 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Miranda Le whose telephone number is (571) 272-41 12. 
The examiner can normally be reached on Monday through Friday from 10:00 AM to 
6:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, James K. Trujillo, can be reached on (571) 272-3677. The fax number to 
this Art Unit is (571)-273-8300. 

Any inquiry of a general nature or relating to the status of this application should 
be directed to the Group receptionist whose telephone number is (571) 272-2100. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see <http://pair-direct.uspto.gov>. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

/Miranda Le/ 
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